Apparatus and methods for aggregation of communication bandwidth over multiple wireless communication links

ABSTRACT

Fixed-length communication traffic blocks, for transmission over a wireless communication link, are generated from communication traffic that is received at a primary communication apparatus. The generated blocks are distributed between a first wireless communication link associated with the primary apparatus and a second wireless communication link associated with a secondary apparatus, for transmission of the communication traffic blocks over the first and second wireless communication links. The distribution of the communication traffic blocks is through an inter-apparatus communication link between the primary and secondary apparatus. Communication traffic blocks that are received over the second wireless communication link are transferred to the primary apparatus through the inter-apparatus communication ink and combined, with blocks that are received on the first wireless communication link, into a communication traffic stream. One or more secondary wireless communication links may be provided by one or more respective installations of secondary apparatus.

FIELD OF THE INVENTION

This invention relates generally to communications and, in particular,to aggregating bandwidth that is provided by multiple wirelesscommunication links.

BACKGROUND

The growth of Ethernet as a transport medium for voice and data in awide variety of technologies, such as Metro Ethernet Networks, isincreasing the demand for the bandwidth it offers. While the use of thewireless technology in such networks is also growing because of itssimplicity in installation and cost, the limited bandwidth capacity ofwireless communication links is adversely affecting wide scaledeployment of such technology in core networks.

In order to increase bandwidth, multiple wireless links are integratedwith Ethernet switches implementing traditional load sharing orbalancing techniques such as Link Aggregation control Protocol, PVSTP(Per Virtual Local Area Network (VLAN) Spanning Tree Protocol), and MSTP(Multiple STP). However, bandwidth sharing between the wireless linksaccording to such techniques is dependent on the profile of trafficstreams.

SUMMARY

According to an aspect of the invention, an apparatus includes: a blockgenerator that receives communication traffic and generates fixed-lengthcommunication traffic blocks for transmission over a wirelesscommunication link, the communication traffic blocks comprising thereceived communication traffic; a distributor, operatively coupled tothe block generator, that distributes the communication traffic blocksbetween a first wireless communication link associated with theapparatus and a second wireless communication link associated with afurther apparatus for transmission of the communication traffic blocksover the first and second wireless communication links; and a blocktransfer module, operatively coupled to the distributor, that enablesthe communication traffic blocks that are distributed to the secondwireless communication link to be transferred to the further apparatusover a further communication link.

In some embodiments, the block generator receives the communicationtraffic over a communication link, and the communication link and thefurther communication link are of the same type. Where the communicationlink and the further communication link are respective Ethernet links,the block transfer module encapsulates the communication traffic blocksthat are distributed to the second wireless communication link intoEthernet frames.

Although the received communication traffic may include headerinformation, the distributor may distribute the communication trafficblocks between the first wireless communication link and the secondwireless communication link independently of the header information.

The block generator includes a unique identifier in each of thecommunication traffic blocks in some embodiments.

The distributor may distribute the communication traffic blocks to loadbalance transmission of the received communication traffic over thefirst and second wireless communication links, for example.

An interface may be operatively coupled to the block transfer module toenable communications with the further apparatus over the furthercommunication link. The interface may also enable exchange of managementtraffic between the apparatus and the further apparatus.

The distributor may determine an operational status of each of the firstand second wireless communication links, and determine a distribution ofthe communication traffic blocks based on the determined operationalstatus.

In some embodiments, the first wireless communication link is a primarywireless communication link, the second wireless communication link isone of multiple secondary wireless communication links with eachsecondary wireless communication link being associated with a respectivefurther apparatus, the distributor distributes the communication trafficblocks between the first wireless communication link and the secondarywireless communication links, and the block transfer module enables thecommunication traffic blocks that are distributed to the secondarywireless communication links to be transferred to the further apparatuswith which each of the secondary wireless communication links isassociated.

The block transfer module may also enable communication traffic blocksreceived by the further apparatus to be transferred to the apparatusfrom the further apparatus over the second wireless communication link,in which case the apparatus may also include a block combiner,operatively coupled to the block transfer module, that combinescommunication traffic blocks that are received from the furtherapparatus with communication traffic blocks that are received over thefirst wireless communication link into a combined communication trafficstream. Where the communication block generator receives thecommunication traffic over a communication link, the combinedcommunication traffic stream may be for transmission over thecommunication link.

The apparatus may also include a block flow controller, operativelycoupled to the distributor, that controls distribution of thecommunication traffic blocks to the second wireless communication linkby the distributor.

Such an apparatus could be implemented, for example, in a first node ofa communication system, and in combination with a second node of thecommunication system that includes the further apparatus. In this case,the first node may also include a wireless interface that enables thecommunication traffic blocks that are distributed to the first wirelesscommunication link to be transmitted over the first wirelesscommunication link, the further apparatus may include a block transfermodule that enables the communication traffic blocks that aredistributed to the second wireless communication link to be received bythe further apparatus over the further communication link, and thesecond node may further include a wireless interface, operativelycoupled to the block transfer module of the further apparatus, thatenables the communication traffic blocks that are distributed to thesecond wireless communication link to be transmitted over the secondwireless communication link.

In such a combination the further apparatus may include a blockgenerator that receives communication traffic and generates fixed-lengthcommunication traffic blocks comprising the received communicationtraffic; and a distributor, operatively coupled to the block generatorand to the block transfer module, that distributes the communicationtraffic blocks between the first and second wireless communication linksfor transmission. The block transfer module of the further apparatus mayenable the communication traffic blocks that are distributed to thefirst wireless communication link by the distributor of the furtherapparatus to be transferred to the apparatus over the furthercommunication link, and the block transfer module of the apparatus mayenable the communication traffic blocks that are distributed to thefirst wireless communication link by the distributor of the furtherapparatus to be received by the apparatus over the further communicationlink.

In some embodiments, the apparatus of the first node is configurable foroperation in either of a primary operating mode in which the blockgenerator of the apparatus receives communication traffic and generatesfixed-length communication traffic blocks comprising the receivedcommunication traffic and the distributor of the apparatus distributesthe communication traffic blocks between the first and second wirelesscommunication links, and a secondary operating mode in which the blocktransfer module of the apparatus receives communication traffic blocksthat are distributed to the first wireless communication link by thedistributor of the further apparatus. The further apparatus of thesecond node may similarly be configurable for operation in either of theprimary operating mode in which the block generator of the furtherapparatus receives communication traffic and generates fixed-lengthcommunication traffic blocks comprising the received communicationtraffic and the distributor of the further apparatus distributes thecommunication traffic blocks between the second and first wirelesscommunication links, and the secondary operating mode in which the blocktransfer module of the further apparatus receives communication trafficblocks that are distributed to the second wireless communication link bythe distributor of the apparatus.

One of the apparatus and the further apparatus might be configured foroperation in the primary operating mode, with the other of the apparatusand the further apparatus being configured for operation in thesecondary operating mode. The configurations of the apparatus and thefurther apparatus could be adjustable, to configure the one of theapparatus and the further apparatus for operation in the secondaryoperating mode and to configure the other of the apparatus and thefurther apparatus in the primary operating mode.

A method according to another aspect of the invention includes:receiving communication traffic at a first communication apparatus;generating fixed-length communication traffic blocks for transmissionover a wireless communication link, the communication traffic blockscomprising the received communication traffic; distributing thecommunication traffic blocks between a first wireless communication linkassociated with the first communication apparatus and a second wirelesscommunication link associated with a second communication apparatus fortransmission of the communication traffic blocks over the first andsecond wireless communication links; transmitting the communicationtraffic blocks that are distributed to the first wireless communicationlink over the first wireless communication link; and transferring thecommunication traffic blocks that are distributed to the second wirelesscommunication link to the second communication apparatus over a furthercommunication link.

The further communication link might be an Ethernet link, in which casetransferring might involve encapsulating the communication trafficblocks that are distributed to the second wireless communication linkinto Ethernet frames.

Even if the received communication traffic includes header information,distributing may involve distributing the communication traffic blocksbetween the first wireless communication link and the second wirelesscommunication link independently of the header information.

Generating involves including a unique identifier in each of thecommunication traffic blocks in some embodiments.

Although other distribution schemes could potentially be used,distributing involves distributing the communication traffic blocks toload balance transmission of the received communication traffic over thefirst and second wireless communication links in one embodiment.

The first wireless communication link may be a primary wirelesscommunication link, and the second wireless communication link may beone of multiple secondary wireless communication links, with eachsecondary wireless communication link being associated with a respectivefurther apparatus. In this case distributing may involve distributingthe communication traffic blocks between the first wirelesscommunication link and the secondary wireless communication links, andtransferring may involve transferring the communication traffic blocksthat are distributed to the secondary wireless communication links tothe further apparatus with which each of the secondary wirelesscommunication links is associated.

The method may also include: receiving at the first communicationapparatus, from the second communication apparatus over the furthercommunication link, communication traffic blocks received by the secondcommunication apparatus over the second wireless communication link;receiving communication traffic blocks over the first wirelesscommunication link at the first communication apparatus; and combiningcommunication traffic blocks that are received from the secondcommunication apparatus with communication traffic blocks that arereceived over the first wireless communication link into a combinedcommunication traffic stream. Where receiving the communication trafficinvolves receiving the communication traffic over a communication link,the combined communication traffic stream could be for transmission overthe communication link.

In some embodiments, the first communication apparatus, when configuredin a primary operating mode, performs the receiving, generating,distributing, transmitting, and transferring. The second communicationapparatus, when configured in the primary operating mode, may similarlyreceive communication traffic, generate fixed-length communicationtraffic blocks comprising the received communication traffic, distributethe communication traffic blocks between the second and first wirelesscommunication links for transmission, transmit the communication trafficblocks that are distributed to the second wireless communication linkover the second wireless communication link, and transfer thecommunication traffic blocks that are distributed to the first wirelesscommunication link to the first communication apparatus over the furthercommunication link. When configured in a secondary operating mode, thefirst communication apparatus might receive communication traffic blocksthat are distributed to the first wireless communication link by thesecond communication apparatus over the further communication link andtransmit the received communication traffic blocks over the firstwireless communication link. The second communication apparatus, whenconfigured in the secondary operating mode, may similarly receivecommunication traffic blocks that are distributed to the second wirelesscommunication link by the first communication apparatus over the furthercommunication link and transmit the received communication trafficblocks over the second wireless communication link. The method mightthen involve configuring one of the first and second communicationapparatus for operation in the primary operating mode, configuring theother of the first and second communication apparatus for operation inthe secondary operating mode, and adjusting the configurations of thefirst and second communication apparatus, to configure the one of thefirst and second communication apparatus for operation in the secondaryoperating mode and to configure the other of the first and secondcommunication apparatus in the primary operating mode.

Another aspect of the invention provides apparatus that includes: aninterface that enables reception of communication traffic blocks over afirst wireless communication link associated with the apparatus; a blocktransfer module that enables communication traffic blocks, that arereceived by a further apparatus over a second wireless communicationlink associated with the further apparatus, to be transferred to theapparatus from the further apparatus over a further communication link;and a block combiner, operatively coupled to the interface and to theblock transfer module, that combines communication traffic blocks thatare received from the further apparatus with communication trafficblocks that are received over the first wireless communication link intoa combined communication traffic stream.

A related method includes: receiving communication traffic blocks over afirst wireless communication link associated with a first communicationapparatus; receiving at the first communication apparatus, from a secondcommunication apparatus over a further communication link, communicationtraffic blocks that are received by the second communication apparatusover a second wireless communication link associated with the secondcommunication apparatus; and combining communication traffic blocks thatare received from the second communication apparatus and that comprisecommunication traffic of a communication traffic stream withcommunication traffic blocks that are received over the first wirelesscommunication link and that comprise communication traffic of thecommunication traffic stream into a combined communication trafficstream.

There is also provided an apparatus including: an interface that enablescommunication of communication traffic blocks over a first wirelesscommunication link; and a block transfer module, operatively coupled tothe interface, that enables transfer of communication traffic blocksbetween the apparatus and a further apparatus over a furthercommunication link, the communication traffic blocks and furthercommunication traffic blocks that are communicated over a secondwireless communication link associated with the further apparatustogether comprising communication traffic of a communication trafficstream.

Yet another aspect of the invention provides a method that includes:communicating communication traffic blocks over a first wirelesscommunication link associated with a first communication apparatus; andtransferring the communication traffic blocks between the firstcommunication apparatus and a second communication apparatus over afurther communication link, the communication traffic blocks and furthercommunication traffic blocks that are communicated over a secondwireless communication link associated with the second communicationapparatus together comprising communication traffic of a communicationtraffic stream.

Other aspects and features of embodiments of the present invention willbecome apparent to those ordinarily skilled in the art upon review ofthe following description.

BRIEF DESCRIPTION OF THE DRAWINGS

Examples of embodiments of the invention will now be described ingreater detail with reference to the accompanying drawings.

FIG. 1 is a block diagram of an example communication system.

FIG. 2 is a block diagram of a multiple wireless link communicationsystem.

FIG. 3 is a block diagram of an example multiple wireless linkcommunication system according to an embodiment of the presentinvention.

FIG. 4 is a block diagram of an example multiple wireless linkcommunication system having more than two wireless links.

FIG. 5 is a block diagram of an example block transfer format.

FIGS. 6 to 8 are flow diagrams illustrating methods of embodiments ofthe invention.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of a communication system 10, which includesaccess networks 12, 14, 16, illustratively LANs (Local Area Networks).Switching/routing nodes 20, 22, 24 that are at least coupled to or formpart of the access networks 12, 14, 16 enable communications between theuser networks through a core network. A management system 18, throughwhich various functions of the switching/routing nodes 20, 22, 24 suchas QoS (Quality of Service) can be configured and managed, is alsoshown.

For a wireless ring/mesh network implementation, the switching/routingnodes 20, 22, 24 are operatively coupled to respective wireless linknodes 25, 26, 27, 28, 30, 32. The wireless link nodes 25, 26, 27, 28,30, 32 enable the switching/routing nodes 20, 22, 24, and thus theaccess networks 12, 14, 16 to communicate over wireless communicationlinks. FIG. 1 also shows wireless link nodes 34, 36, to illustrate thatwireless communication links might be used not only in the core network,but also in the access networks 12, 14, 16.

Although the access networks 12, 14, 16 may include many nodes, only oneswitching/routing node 20, 22, 24 per network is shown in FIG. 1 toavoid overly complicating the drawing. It will also be appreciated thata core network could interconnect many more than three switching/routingnodes.

Those skilled in the art will be familiar with various communicationsystems having a general structure similar to that of the system 10, thetypes of communication equipment such as the switching/routing nodes 20,22, 24 provided in such systems, and the operation thereof. The system10 might be a simple Ethernet wired mesh/ring network with the threeswitching/routing nodes 20, 22, 24 interconnecting the different accessnetworks 12, 14, 16, for instance. Various types of wirelesscommunication equipment which could be deployed as the wireless linknodes 25, 26, 27, 28, 30, 32, 34, 36 will also be known to those skilledin the art. Embodiments of the present invention relate primarily toaggregating bandwidth over multiple wireless communication links and isnot specific to any particular type of communication system, network, orequipment.

In one possible implementation, the interfaces to the switching/routingnodes 20, 22, 24, at both the access side and the core network side, areEthernet interfaces which run at 10/100/1000 Mbps. However, thebandwidth offered by each of the wireless communication links isaffected by such parameters as allocated radio channel capacity and themode of operation of wireless modems that support the wireless links.Different signal encoding mechanisms applied by a wireless modem mightprovide different bandwidths, for example. The wireless communicationlinks provided by the wireless link nodes 25, 26, 27, 28, 30, 32 mighttherefore have more limited bandwidth capacity than the Ethernet linksin this example. In one embodiment, the wireless communication linksoperate at 371 Mbps.

The allocated radio channel width and modem mode operation are decidedby a number of factors, possibly including a regulating authority. Thebandwidth offered by the wireless communication links often does notmatch that of the Ethernet links, and is usually less than that of theEthernet links as indicated in the above example.

One way of addressing the issue of lower bandwidth on the wirelesscommunication links involves using Link Aggregation control Protocol ora VLAN based spanning tree protocol. These techniques could be appliedin a multiple wireless link communication system such as shown in FIG.2.

The system 40 of FIG. 2 includes two access networks 42, 44, whichcommunicate via switching/routing nodes 46, 48 and wirelesscommunication links provided by the wireless link nodes 52/56, 54/58. Inthis system, the wireless bandwidth between the switching/routing nodes46, 48 is increased by providing two wireless communication links,instead of one, between those switching/routing nodes.

The switching/routing nodes 46, 48 can run the Link Aggregation controlProtocol between their respective wireless link nodes 52/54, 56/58. Atthe switching/routing node 46, for example, the two ports or interfacesto which the wireless link nodes 52, 54 are connected are groupedtogether and traffic is distributed amongst them according to standarddistribution algorithms. The same grouping and distribution algorithmsare applied by the switching/routing node 48 to the two ports orinterfaces to which the wireless link nodes 56, 58 are connected.Another way of handling the multiple wireless communication links in thesystem 40 is to configure the two switching/routing nodes 46, 48 todistribute traffic based on VLAN ids using a Spanning Tree Protocol.

In both of these cases true bandwidth distribution cannot be achieved,since these schemes distribute the traffic based on traffic streamcharacteristics. Although two different traffic streams or flows mightbe distributed to the two wireless links by the switching/routing nodes46, 48, a traffic stream or flow that is directed to a particulardestination from a particular source, for example, would not be “split”between the two wireless communication links. There is no way toguarantee that the traffic streams or flows in a network are balanced interms of their bandwidth requirements.

Another possible issue that may arise with evenly splitting a trafficflow is variable frame or packet sizes that are used in manycommunication protocols. Ethernet frames, for example, are ofvariable-length, and therefore received Ethernet frames cannot be evenlydistributed between multiple wireless communication links unless thereceived frames have the same length, which cannot be guaranteed.

Embodiments of the invention provide a new scheme for aggregatingbandwidth. Multiple wireless links are interconnected for aggregatingwireless bandwidth. Incoming communication traffic is processed anddistributed among the interconnected wireless links. Proportionalbandwidth sharing and hence increased bandwidth utilization can beachieved immaterial of the incoming traffic profile. The proposedschemes may enable wide scale deployment of wireless communication linksin Ethernet networks, for example, while alleviating concerns regardingbandwidth under-utilization.

The presently disclosed mechanisms help in aggregating the bandwidth ofmultiple wireless links. A primary node or apparatus interfaces with anetwork, illustratively a user network, processes user traffic, andcreates multiple streams of radio blocks, for both primary and secondarywireless communication links. The primary node sends radio blocks to andreceives radio blocks from the secondary node(s), and recreates usertraffic streams. The primary node distributes user data among themultiple links independent of the actual content of received traffic,thus achieving proportional load balancing.

FIG. 3 is a block diagram of a multiple wireless link communicationsystem according to an embodiment of the present invention. The system60 includes two communication apparatus 62, 64. The apparatus 62includes a network interface 66, a wireless interface 68, an apparatusinterface 70, and a management interface 72. A block generator 74, adistributor 76, a block combiner 78, a block transfer module 80, a blockflow controller 81, and a controller 82 are operatively coupled to theinterfaces 66, 68, 70, 72 and to each other as shown. The apparatus 64includes an apparatus interface 84, a management interface 86, awireless interface 88, a block transfer module 90 operatively coupled tothe apparatus interface and to the wireless interface, a flow commandencoder 91 operatively coupled to the wireless interface and to theblock transfer module, and a controller 92 operatively coupled to themanagement interface and possibly to other apparatus components. Thedashed lines in FIG. 3 illustrate optional connections, which might beprovided in one particular implementation of a management scheme asdescribed in further detail below.

Communication equipment in which the example apparatus 62, 64 areimplemented, such as different wireless link nodes, may includeadditional components that have not been explicitly shown in FIG. 3 inorder to avoid overly complicating the drawing. More generally, otherembodiments may include further, fewer, or different components whichmay be interconnected in a similar or different manner than shown.

The network interface 66 includes components which supportcommunications over a network communication link, illustratively a linkto an Ethernet switching/routing node. Such components often includehardware at least in the form of a physical port or connector. Trafficprocessing such as QoS processing and/or rate limiting, for example, mayalso be performed by the network interface 66. The network interface 66is intended to represent a module that handles communication trafficthat is received by the apparatus 62. Thus, the network interface 66 mayinclude other components, illustratively at least a transceiver and apacket processor for instance, in addition to a physical port orconnector.

Hardware, firmware, components which execute software, or somecombination thereof might be used in implementing the network interface66, and possibly other elements of the apparatus 62, 64. Electronicdevices that may be suitable for this purpose include, among others,microprocessors, network processors, microcontrollers, PLDs(Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays),ASICs (Application Specific Integrated Circuits), and other types of“intelligent” integrated circuits.

The exact structure of the network interface 66 may, to at least someextent, be implementation-dependent, and could vary depending on thetype of connection(s) and/or protocol(s) to be supported. In oneembodiment, the network interface 66 supports Ethernet communications,in which case the network interface processes received Ethernet framesto extract content for transmission via the wireless links that areprovided by the wireless interfaces 68, 88.

The wireless interfaces 68, 88, the apparatus interfaces 70, 84, and themanagement interfaces 72, 86, like the network interface 66, includecomponents such as physical ports or connectors and possibly othercomponents which support communications over respective communicationlinks. In the case of the wireless interfaces 68, 88, these componentsmay include at least a wireless modem, a radio, and/or other componentsthat are used in communicating over wireless communication links. In oneembodiment, the apparatus interfaces 70, 84 are spare Ethernetinterfaces that are provided in wireless link nodes. The managementinterfaces 86 are also Ethernet interfaces for connection to aswitching/routing node in one embodiment. As noted above for the networkinterface 66, the structures of the wireless interfaces 68, 88, theapparatus interfaces 70, 84, and the management interfaces 72, 86 may bedependent upon the type of connection(s) and/or protocol(s) that are tobe supported.

In many implementations, these interfaces will be identical in theapparatus 62, 64. However, embodiments of the invention are not in anyway limited to such implementations. The mechanisms disclosed hereincould potentially be used in conjunction with wireless interfaces 68, 88that support wireless communication links having different bandwidths,for instance. In one particular embodiment described in detail below,each apparatus 62, 64 includes two Ethernet interfaces in addition to awireless interface 68, 88. In the apparatus 62, one of the Ethernetinterfaces is used as the network interface 66, the other is used as theapparatus interface 70, and the management interface 72 is a “virtual”interface as opposed to a separate physical interface. In the apparatus64, one of the Ethernet interfaces is used as the apparatus interface 84and the other is used as the management interface 86 to enablemanagement traffic to be exchanged with a switching/routing node.

The block generator 74, the distributor 76, the block combiner 78, theblock transfer modules 80, 90, the block flow controller 81, the flowcommand encoder 91, and the controllers 82, 92 may be implemented usinghardware, firmware, and/or components which execute software. Thesemodules are defined moreso by their functions rather than a particularinternal structure. The present disclosure would enable a skilled personto implement these modules in any of various ways to perform theirrespective functions. The block flow controller 81 and the controller 82have been shown separately in FIG. 3 to represent components whichperform different control functions. In one embodiment, the block flowcontroller 81 is involved in controlling the flow of radio blocksbetween primary and secondary apparatus, as described in further detailbelow, whereas the controller 82 performs more general control functionsfor managing each apparatus.

In operation, the apparatus 62 and the apparatus 64 are interconnectedto each other through the apparatus interfaces 70, 84, and togetherprovide two wireless links. A similar installation of interconnectedapparatus would be provided at the other end of the wirelesscommunication links. This type of installation could be used to increasethe overall capacity on any of the wireless links shown in FIG. 1, forexample. Additional wireless link nodes could be connected to thewireless link nodes 25, 30 to increase wireless bandwidth between theswitching/routing nodes 20, 24. The wireless bandwidth between othercomponents of the system 10 could be increased in a similar manner.

As described in further detail herein, communication traffic received bythe network interface 66 is processed and converted into a radio blockstream. The radio blocks are then distributed between the two wirelesscommunication links. The radio blocks to that are distributed to thewireless communication link that is associated with the apparatus 64 andspecifically the wireless interface 88 are delivered over theinter-apparatus communication link, illustratively an Ethernetconnection, through the apparatus interfaces 70, 84.

In one embodiment, each wireless communication link, while working onits own, has fixed wireless bandwidth. When configured to work in thebandwidth aggregation mode, one of the wireless communication linksmight be a primary link, with the other being a secondary link. In thesystem 60, the wireless communication link associated with the apparatus62 is the primary link, and the wireless communication link associatedwith the apparatus 64 is the secondary link. The apparatus 64 does notdirectly process received communication traffic, but rather acts as awireless conduit. The primary apparatus 62 connects to aswitching/routing node, for example, through the network interface 66.Received communication traffic might be Ethernet frames carrying userEthernet data, for instance. The apparatus 62, and in particular itsnetwork interface 66, may classify and prioritize the data if QoS isenabled, and other processing may also be performed.

The received traffic is then converted into appropriate radio blocks fortransmission on a wireless communication link and distributed over thelocal and secondary wireless communication links. The radio blocks to besent to the secondary wireless communication link are encapsulated inEthernet headers in one embodiment for transfer over an Ethernetconnection between the apparatus interfaces 70, 84. In the case of anEthernet connection between the primary apparatus 62 and the secondaryapparatus 64, the Ethernet headers that were used to transport the radioblocks are removed and the radio blocks are sent over the wirelessinterface 88.

At the other end of the wireless communication links, the radio blocksare combined to reproduce the originally received communication traffic.

In the other direction, radio blocks that are received through thewireless interface 88 in the secondary apparatus 64 are encapsulated inEthernet headers and sent to the primary apparatus 60, where theEthernet headers are removed. The resulting radio blocks are combinedwith radio blocks arriving through the wireless interface 68 to delivercommunication traffic to the network interface 66 for transmission fromthe primary apparatus 62.

It can be noted from this example that the bandwidths of two wirelesscommunication links can be combined to service the traffic from one portor connection from a switching/routing node to which the networkinterface of the apparatus 62 is connected. This mechanism gives theflexibility to process communication traffic in the apparatus 62 in sucha way that the traffic can be subjected to the same QoS mechanism and/orother traffic processing, and one stream of radio blocks is produced.These radio blocks are then distributed among the two wirelesscommunication links independent of the characteristics of the trafficthat was used to generate those blocks.

Considering the example system 60 and its operation in more detail, theblock generator 74 receives communication traffic that has beenprocessed by the network interface 66 and generates fixed-lengthcommunication traffic blocks for transmission over a wirelesscommunication link. The generated communication traffic blocks,described above as radio blocks, might include user data from thereceived communication traffic. The distributor 76 distributes thecommunication traffic blocks between a first wireless communication linkassociated with the apparatus 62 and a second wireless communicationlink associated with the apparatus 64 for transmission over the firstand second wireless communication links. The block transfer module 80enables the communication traffic blocks that are distributed to thesecond wireless communication link to be transferred to the apparatus 64over the inter-apparatus communication link, through the apparatusinterface 70 in the example shown.

In some embodiments, the communication link over which the communicationtraffic is received and the inter-apparatus communication link are ofthe same type, although different types of links are also contemplated.

Where the inter-apparatus communication link is an Ethernet link, theblock transfer module 80 might encapsulate the communication trafficblocks that are distributed to the second wireless communication linkinto Ethernet frames.

As noted above, some traffic distribution mechanisms do not distributethe same traffic flow to different wireless links. In the example system60, however, the distributor 76 may distribute the communication trafficblocks that are generated by the block generator 74 between the firstwireless communication link and the second wireless communication linkindependently of source information and destination information in thereceived communication traffic. The distribution may also be independentof other header or overhead information in the received communicationtraffic, to provide for true bandwidth sharing or load balancing oftransmission of the received traffic over the first and second wirelesscommunication links.

Communication traffic that is received by through the network interface66 is thus separated into a sequence of radio blocks by the blockgenerator 74. Preserving the original order of received communicationtraffic in the sequence of radio blocks simplifies the process ofrebuilding the radio blocks at a receiving end of the wirelesscommunication links, since the original communication traffic can thenbe rebuilt by combining the contents of the radio blocks in theirsequential order, as described in further detail below.

Although embodiments of the present invention may involve distributionof communication traffic independently of traffic profiles or overheadinformation, any of various parameters may be taken into account indetermining the exact distribution that is to be applied. For example,the distributor 76 might determine an operational status of each of thefirst and second wireless communication links, and determine adistribution of communication traffic blocks based on the determinedoperational status. Where one of the wireless communication links failsor an error rate observed on one of those links is higher than athreshold, for instance, the distributor 76 could route communicationtraffic blocks only to the other link. This provides a level ofredundancy protection. The distributor 76 could also or insteaddetermine data rates on the wireless communication links and set adistribution ratio accordingly. A block flow control mechanism may alsoor instead be provided between a primary apparatus and any secondaryapparatus, and an example of such a mechanism using the block flowcontroller 81 and the flow command encoder 91 is described in detailbelow.

Parameters that affect the distribution of communication traffic blocksneed not necessarily be monitored by the distributor 76 directly. Thecontroller 82 or some other component(s) of the apparatus 62 or anetwork node in which or in conjunction with which the apparatus isimplemented might monitor and report any such parameters, or providecommands to vary the distribution mechanism on the basis of theparameters, to the distributor 76. Explicit management of thedistribution mechanism by a network operator or other personnel, throughthe management interface 72 and the controller 82 for instance, is alsopossible.

In order to assist a receiving apparatus at the other end of thewireless communication links in regenerating the original receivedcommunication traffic stream from the traffic blocks it receives overthe wireless links, the block generator 74 might include a uniqueidentifier in each of the communication traffic blocks. Such anidentifier could be a sequence number, for example, which indicates anorder of a block relative to other blocks that were generated from thesame traffic stream, for example. As noted above, preserving the orderof a received communication traffic stream in the blocks that aregenerated by the block generator 74 allows a receiving apparatus toregenerate the traffic stream by combining received blocks in theirsequential order, without looking into other content of those blocks todetermine how the blocks are to be combined. In one embodiment, sequencenumbers that are added to the blocks by the block generator 74, and areused at the receiving end to properly sequence the blocks for combining.

The system 60 may also receive communication traffic over the first andsecond wireless communication links. To this end, the block transfermodule 80 also enables communication traffic blocks received by theapparatus 64 through the wireless interface 88 to be transferred to theapparatus 62. The block combiner 78 combines communication trafficblocks that are received from the apparatus 64 with communicationtraffic blocks that are received over the first wireless communicationlink through the wireless interface 68, in the order indicated by theirsequence numbers for instance. This combining results in a combinedcommunication traffic stream that regenerates a communication trafficstream that was received at the other end of the wireless communicationlinks and transmitted to the system 60 over those links. The combinedcommunication traffic stream would then be provided to the networkinterface 66 for transmission.

FIG. 3 shows the primary apparatus 62 and the secondary apparatus 64 ashaving different implementations, with the secondary apparatus havingonly a subset of the components of the primary apparatus. In someembodiments, however, the primary and secondary apparatus are identicalin structure, and are configurable for operation in either of a primaryoperating mode and a secondary operating mode. When configured in theprimary operating mode, the block generator of an apparatus receivescommunication traffic and generates fixed-length communication trafficblocks, and the distributor distributes the generated communicationtraffic blocks. The block transfer module of an apparatus that isconfigured to operate in the secondary operating mode receivescommunication traffic blocks that are distributed to the wirelesscommunication link of the apparatus by the distributor of anotherapparatus. The block generator, the distributor, and the block combiner,as well as the block flow controller described below, in an apparatusthat is operating in secondary mode might simply be inactive.

Normally only one apparatus of an interconnected group would beconfigured for operation in the primary operating mode and the otherwould be configured for operation in the secondary operating mode. Theseconfigurations could potentially be adjustable, illustratively toreverse the primary/secondary designations. This type of functionalitycould be supported where the apparatus 64 also has a network interface66, a block generator 74, a distributor 76, and a block combiner 78.

Adjustments to primary/secondary configurations could beoperator-driven, through the management interfaces 72, 86. In onepossible implementation, the apparatus 62, 64 are connected to the sameswitching/routing node through their management interfaces 72, 86, toenable management information such as control or configurationinformation to be exchanged with a remote management or control systemsuch as the management system 18 shown in FIG. 1.

Automatic switching of primary/secondary configurations is alsopossible. Where the apparatus 64 has a network interface and a separatecommunication link to the same network as the apparatus 62, then theapparatus 64 could be reconfigured as the primary apparatus, to exchangecommunication traffic through the interface that is being used as themanagement interface 86 in one embodiment, if the network communicationlink of the apparatus 62 fails. Failure of a network communication linkcould be detected, and a configuration switch could also be initiated,by one or more of the primary apparatus, the network, or possibly thesecondary apparatus. One or both of the management interfaces 72, 86could be used to coordinate a primary/secondary reconfiguration.

A secondary apparatus could also or instead be reconfigured as a primaryapparatus under other conditions as well. For example, a fault orfailure affecting the primary apparatus, and not just its networkcommunication link, could lead to a secondary apparatus beingreconfigured as the primary apparatus. Other conditions that mightautomatically initiate or otherwise lead to a primary/secondaryreconfiguration may be or become apparent to those skilled in the art.

Management traffic could potentially be exchanged over the same links ascommunication traffic in some embodiments. For example, the apparatusinterfaces 70, 84 could also enable exchange of management trafficbetween the apparatus 62, 64 for managing the first and second wirelesscommunication links and/or other features or components of theapparatus. Tags or other identifiers could be applied to managementtraffic packets or information by the block generator 74 or the blocktransfer modules 80, 90, for example, to differentiate communicationtraffic blocks from management traffic. Received management traffic canthen be provided to the controllers 82, 92. The management interfaces72, 86 could be provided even where management traffic is exchangedbetween interconnected apparatus 62, 64. Inter-apparatus exchange ofmanagement traffic through the apparatus interfaces 70, 84 might be usedby the primary and secondary apparatus to coordinate their operationwith each other, for example, whereas the management interfaces 72, 86would permit other management functions to be performed remotely.

The dashed lines in FIG. 3 illustrate another possible managementscheme. In one embodiment, each apparatus 62, 64 includes two Ethernetinterfaces in addition to its wireless interface 68, 88. These Ethernetinterfaces in the apparatus 62 are respectively used as the networkinterface 66 and the apparatus interface 70, and the managementinterface 72 is effectively a virtual interface rather than a separateexternal interface. The two Ethernet interfaces in the apparatus 64 arerespectively used as the apparatus interface 84 and the managementinterface 86, which is operatively coupled to the same switching/routingnode as the network interface 66 but handles management traffic.

With this type of implementation, the secondary apparatus 64, theprimary apparatus 62, and the counterpart or peer primary and secondaryapparatus at the other end of the wireless communication links can allbe managed through the management interface 86. Management traffic isreceived by the management interface 86 and provided to the localcontroller 92 if that management traffic is to be consumed by the localcontroller, or to the apparatus interface 84 in the example shown if thereceived management traffic is for another controller. In some cases,management traffic could potentially be applied to all managedcontrollers, and thus could be provided to both the local controller 92and the apparatus interface 84.

Management traffic that is to be transferred to another controller issent to the apparatus 62 through the apparatus interfaces 84, 70. In theabove example of Ethernet interfaces as the interfaces 66, 70, 84, 86,the received management traffic is Ethernet traffic and thus themanagement traffic might not need further processing for transferbetween the apparatus 64, 62. The management traffic received at theapparatus interface 70 is provided to the controller 82 through themanagement interface 72 and/or to the network interface 66. This enablesthe controller 82 to also be managed through the management interface 86of the secondary apparatus 64 and a virtual management interface 72,rather than two separate physical management interfaces.

Any management traffic that is received by the network interface 66 canbe processed in the same manner as communication traffic, for transferto the other end of the wireless communication links. Radio blocksincluding the management traffic are generated by the block generator74, which could include tags or other information in the generatedblocks to allow those blocks to be identified as blocks that includemanagement traffic. The generated blocks are distributed by thedistributor 76, and transmitted over the wireless communication links.

At the receiving end of the wireless links, the radio blocks thatinclude the management traffic are received through the wirelessinterfaces 68, 88 and provided to the primary and/or secondarycontrollers 82, 92. Although direct dashed connections between thewireless interfaces 68, 88 and the management interfaces 72, 86 areshown in FIG. 3, these dashed connections are intended to represent alogical flow of management traffic rather than direct physical flow. Inone embodiment, radio blocks that are received through the wirelessinterfaces 68, 88 and include management traffic are actually providedto the block combiner to regenerate the management traffic. Theregenerated management traffic can then be provided to the controller 82through the apparatus interface 70 and the management interface 72,and/or to the controller 92 through the apparatus interfaces 70, 84 andthe management interface 86. Management traffic could potentially betransmitted externally from the secondary apparatus 64 through themanagement interface 86.

In this example, the management interface 86 and the apparatusinterfaces 70, 84 implement such functions as distinguishing betweenradio blocks and management traffic that is received at the managementinterface 86, and providing the received management traffic to a localcontroller and/or to another interface. Where the management interface86, the apparatus interfaces 84, 70, and the network interface 66 areall Ethernet interfaces, or more generally the same type of interface,received management traffic can be transferred between those interfaceswithout further processing. In other embodiments, format conversionsand/or other management traffic processing functions could be performedby the block transfer modules 80, 90 or other components. Providingreceived management traffic to the network interface 66 as describedabove allows the received management traffic to be transmitted to theapparatus at the other end of the wireless communication links as well.

This type of mechanism may be used to manage primary and secondaryapparatus at both ends of the wireless communication links through asingle external management interface, which is the management interface86 in the above example. In some embodiments, an external managementinterface is available in the secondary apparatus at each end of thewireless communication links, and one or both of the external managementinterfaces may be used in managing primary and secondary apparatus.

An example synchronization or block flow control mechanism is shownseparately from the more generic control mechanism represented by thecontrollers 82, 92 in FIG. 3. The primary wireless communication linkand the secondary wireless communication link in the example system 60,or each secondary link where multiple secondary links are provided, maycarry data in a periodic manner and at a predetermined nominal rate.However, the wireless links may operate asynchronously to each other,such that the actual instantaneous rates are close, but not necessarilyidentical. There may be some relative drift between the wireless links,arising from tolerance and stability characteristics unique to eachapparatus, for example.

Due to the periodic nature of the wireless communication links in someembodiments, each link can be thought of as presenting a series oftransmission opportunities, or “slots”. In order to make the best use ofover-air bandwidth, it may be desirable to have the distributor 76allocate radio blocks to all of these slots. If radio blocks are to beallocated to the wireless communication links on a dynamic basis, thenthe distributor 76 continually adapts in order to meet the “needs”,i.e., fully utilize the available bandwidth, of each link. To achievethis, real-time information about the availability of upcomingtransmission slots on each secondary wireless communication link isprovided to the distributor 76 through the inter-apparatus connectionbetween the apparatus interfaces 70, 84.

Other traffic that shares the interconnecting medium, such as managementtraffic as described above, may make it difficult to communicate eventsin a predictable and timely manner. Instead of attempting to signal theprimary apparatus to transfer each individual radio block for asecondary wireless communication link, which could potentially be donein some embodiments, according to the example shown in FIG. 3 a queue ofradio blocks waiting to be transmitted is established for each link,illustratively in the wireless interfaces 68, 88. The length of thetransmit queue for each wireless communication link is monitored, and ablock flow control mechanism attempts to keep the length of each queuebetween upper and lower thresholds. Since the distributor 76 isco-located with the primary wireless communication link, it has directvisibility into the queue of radio blocks awaiting transmission overthat link. However, there is no such direct visibility into the queue(s)for the secondary link(s).

In one embodiment, management of adaptive distribution of radio blocksin this environment takes advantage of the fact that the rates aresimilar among links. The distributor 76 can then effectively assume thatthe or each secondary wireless communication link requires radio blocksat the same rate as the primary wireless communication link, anddistributes the radio blocks accordingly. At a secondary apparatus, therequirement for more or fewer radio blocks for its secondary link isdetermined from the queue of transmit radio blocks. For example, if thequeue is longer than a predetermined threshold, then it could be judgedthat fewer blocks per second or other time frame are required, and ifthe queue length is shorter than a second threshold, then it could bejudged that more radio blocks are required. The queue length relative tothe thresholds is encoded into flow control or management commands bythe flow command encoder 91 and transferred to the primary apparatus 62.In the system 60, flow commands are transferred to the primary apparatus62 through the block transfer module 90, which encapsulates the flowcommands into Ethernet frames in one embodiment. Depending on the formatof the flow commands and the inter-apparatus communication link, thistransfer could involve further or different processing, or possibly noadditional processing by the block transfer module or other componentsprior to transfer through the apparatus interface 84. Thus, the flowcommand encoder 91 could be operatively coupled to the apparatusinterface 84 instead of to the block transfer module 90.

In one embodiment, the monitored queue length is encoded by the flowcommand encoder into one of 3 flow management commands: NOP, SEND_MOREand SEND_LESS. These commands are transferred to the primary apparatusover the shared inter-apparatus link to the block flow controller 81. Asnoted above for the flow command encoder 91, the block flow controller81 need not necessarily receive flow commands through the block transfermodule 80 as shown in the example system 60.

The block flow controller 81 may implement a flow control engine orprocess for each secondary wireless communication link. In oneembodiment, each flow control engine can exhibit one of 8 states, simplydesignated with integer numbers: {3, 2, 1, 0, −1, −2, −3, −4}. When theengine state is non-negative, i.e., one of {3, 2, 1, 0}, the block flowcontroller 81 provides a command the distributor 76 to allocate a radioblock to the associated secondary wireless communication link at thenext opportunity. These commands may be the same as the flow commandsreceived from the secondary apparatus 64 or of a different form. Thedistributor 76 serves such commands for each secondary wirelesscommunication link in a round-robin fashion in some embodiments.

In this example, the states of all flow control engines could start outat 0, and be decremented each time a radio block is allocated to theassociated secondary wireless communication link. Initially on startupthen, each secondary wireless communication link is allocated 1 radioblock, and the state of its associated flow control engine subsequentlybecomes −1. The states of all flow control engines for the secondarywireless communication links could also be incremented whenever theprimary wireless communication link is allocated a radio block. Thus, inthe absence of any other mechanism in this example, when the primarywireless communication link is allocated a radio block, the states forall flow control engines become 0 and all secondary wirelesscommunication links are allocated a radio block, after which their flowcontrol engine states return to −1.

When the block flow controller 81 receives a flow command for asecondary wireless communication link, the state of the flow controlengine for that link is adjusted accordingly. On receiving a SEND_MOREcommand, the flow control engine increments its state, and on receivinga SEND_LESS command, it decrements its state. In this manner, aSEND_MORE command will allow the distributor 76 to send an extra radioblock to a secondary wireless communication link relative to the numberof radio blocks it sends to the primary wireless communication link, anda SEND_LESS command can prevent the distributor from sending a radioblock to a link that it would have otherwise sent, relative to theprimary link. A NOP command has no effect on the radio blockdistribution and serves only as a placeholder in the flow controlcommand path in one embodiment.

As noted above, 8 flow control engine states are supported in oneembodiment. Although only states 0 and −1 have been explicitly describedabove, the extra states allow for a flow control engine state to beincremented or decremented multiple times within a block allocationcycle. For example, if the block flow controller 81 receives a SEND_LESScommand for a secondary wireless communication link just as a block isallocated to that secondary link, then the state of the flow controlengine for the secondary link is decremented twice. Similarly, if theblock flow controller 81 receives a SEND_MORE command for a secondarywireless communication link just as a block is allocated on the primarywireless communication link, then the state of the flow control engineis incremented twice. Subsequent flow commands may similarly cause flowcontrol engines to transition into higher or lower states.

Embodiments of the invention are not in any way restricted to aparticular number of states, or even to this type of block flow controlmechanism. The process for each flow control engine eventually saturatesat the 3 state or the −4 state in the above example, which may provideadequate block flow control in some applications. Providing a differentnumber of states would enable different saturation and flow controlcharacteristics. Another embodiment uses two-bit states, for example,which supports states of {1, 0, −1, −2} and saturates at either the 1 or−2 level.

The command and radio block allocation or distribution scheme runscontinuously, and manages the flow of radio blocks such that allwireless communication links are fully utilized, and no radio blocks aresent to wireless communication links that are not ready to process them.

It should be appreciated that other block flow control and allocation ordistribution mechanisms are also possible. For instance, although theabove example refers to incrementing flow control engine states forsecondary wireless communication links each time a radio block isdistributed to the primary wireless communication link, therebyeffectively using the primary link and its local timebase to pace blockdistribution to the secondary links, other embodiments could potentiallyuse a different timing source and/or mechanism to control block flow.

Another possible variation of the system 60 and the operations describedabove would be to provide multiple secondary apparatus installations andthus multiple secondary wireless communication links. The distributor 76could then distribute communication traffic blocks between its localwireless communication link and any or all of the multiple secondarywireless communication links. In this case, the block transfer module 80would enable the communication traffic blocks that are distributed tothe secondary wireless communication links to be transferred over arespective further communication link to the secondary apparatus withwhich each of the secondary wireless communication links is associated.

The example described above refers to respective communication linksbetween a primary apparatus and more than one secondary apparatus. Otherimplementations are also contemplated. FIG. 4 is a block diagram ofanother example multiple wireless link communication system having morethan two wireless links. In the example system 100, a primary apparatus102 and two secondary apparatus 104, 106 each have two externalinterfaces, in the form of ports 112/114, 116/118, 120/122, and theseports are connected together in a Daisy chain configuration. One port112 of the primary apparatus 102 is connected to receive and transmitcommunication traffic, and its other port 114 is connected to one of theports 116 of the secondary apparatus 104 through a connection 108. Theother port 118 of the secondary apparatus 104 is connected to one of theports 120 of the secondary apparatus 106 through a connection 110. Theother port 122 of the secondary apparatus 106 is connected to at leastreceive, and possibly transmit, management traffic.

The apparatus 102, 104, 106 may be identical in structure, as describedabove with reference to FIG. 3 and the example system 60, although theirconfigurations and the uses of their respective ports may be different.The primary apparatus 102 and the “last” secondary apparatus 106 areanalogous to the apparatus 62 and the apparatus 64 (FIG. 3),respectively. In the intermediate secondary apparatus 104, the two ports116, 118 are both used as apparatus interfaces for transfer ofcommunication traffic, and management traffic in some embodiments, toand from the primary apparatus 102 and the secondary apparatus 106.

The example system 100 provides three wireless communication links towhich communication traffic can be distributed. In this case, radioblocks that are generated by the primary apparatus 102 are transferredto the secondary apparatus 104 through the same communication link 108.Radio blocks that are to be transmitted over the wireless communicationlink associated with the secondary apparatus 106 are in turn transferredto the secondary apparatus 106, illustratively by the block transfermodule of the secondary apparatus 104. The block transfer module of thesecondary apparatus 104 might determine that a radio block that itreceives from the primary apparatus 102 is to be transferred onward tothe secondary apparatus 106 in accordance with any of variousdistribution schemes. For example, the block generator in the primaryapparatus 102 could include in each distributed radio block anidentifier of the secondary apparatus which is to transmit the radioblock over its wireless communication link. Another option would be fora block transfer module in the primary apparatus 102 to includedifferent destination addresses in Ethernet frames, for example, thatinclude transferred radio blocks. This would enable the secondaryapparatus 104 to determine whether the transferred radio blocks are forits own processing or for further transfer to the secondary apparatus106. A fixed distribution scheme might also or instead be used. In theexample system 100, there are two secondary apparatus 104, 106, and thusthe intermediate secondary apparatus 104 could transfer every secondreceived radio block to the secondary apparatus 106.

FIG. 3 shows an example of a two wireless link system, and FIG. 4 showsan example of a three wireless link system. Higher numbers of wirelesslinks could also be provided by adding further intermediate secondaryapparatus 104 between the primary apparatus 102 and an end secondaryapparatus 106 in a Daisy chain configuration, or multipleinter-apparatus links in a star configuration, for example.

Thus, embodiments of the invention are not limited to arrangements thatprovide only a pair of wireless communication links.

As noted above, radio blocks are encapsulated into Ethernet frames forinter-apparatus transfer in one embodiment. FIG. 5 is a block diagram ofan example block transfer format. The example shown in FIG. 5 is anEthernet frame 130, which includes a destination address 132 to identifythe secondary apparatus that is to process a transferred radio block, aradio block 134, and an FCS (Frame Check Sequence) 136. The destinationaddress 132 could be part of an Ethernet header that is added to theradio block 134 by a block transfer module of a primary apparatus. TheFCS 136 could similarly be added to the radio block 134 in anencapsulation process.

The radio block 134 includes a radio block header 138 and traffic data139. The radio block header could include such information as the uniqueidentifier and/or sequence number described above. Information to allowdelineation of frame boundaries in originally received communicationtraffic from which the radio block was generated could also be includedin the radio block header 138. The traffic data 139 includes actualcontent from the received communication traffic.

Other formats than those shown in FIG. 5 may be used for radio blocksand/or to transfer those blocks between primary and secondary apparatus.The example provided in FIG. 5 is intended solely for the purposes ofillustration.

Aspects of the invention may also or instead be embodied in methods.FIGS. 6 to 8 are flow diagrams illustrating examples of such methods.

With reference to FIG. 6, the method 140 involves receivingcommunication traffic at a first communication apparatus, at 142.Fixed-length communication traffic blocks are generated from thereceived communication traffic at 144 for transmission over a wirelesscommunication link. At 146, the generated communication traffic blocksare distributed between a first wireless communication link associatedwith the first communication apparatus and a second wirelesscommunication link associated with a second communication apparatus fortransmission of the communication traffic blocks over the first andsecond wireless communication links. The communication traffic blocksthat are distributed to the first wireless communication link aretransmitted over the first wireless communication link, and thecommunication traffic blocks that are distributed to the second wirelesscommunication link are transferred to the second communication apparatusover a further communication link, as shown at 148.

FIG. 6 also shows operations that might be involved in controlling radioblock flow. A transmit queue length for a secondary wirelesscommunication link is determined at 150, encoded into a flow command at152, and transferred to the primary apparatus at 154 for use indistributing radio blocks at 146. The operations at 150, 152, 154 may beperformed for each secondary wireless communication link where multiplesecondary links are provided. A local timebase is generated by theprimary apparatus at 156 and is also used in block distribution at 146in the example shown.

The method 160 of FIG. 7 illustrates operations that are performed at areceiving end of aggregated wireless communication links in oneembodiment. Communication traffic blocks are received at 162. Thisincludes not only receiving communication traffic blocks over a firstwireless communication link associated with a first communicationapparatus, but also receiving at the first communication apparatus, froma second communication apparatus over a further communication link,communication traffic blocks that are received by the secondcommunication apparatus over a second wireless communication linkassociated with the second communication apparatus. At 164,communication traffic blocks that are received from the secondcommunication apparatus are combined with communication traffic blocksthat are received over the first wireless communication link, in theircorrect sequential order. A resultant combined communication trafficstream is transmitted at 166.

FIG. 8 illustrates an example of a method 170 that might be performed ata secondary apparatus. In general, communication traffic blocks arecommunicated over a first wireless communication link associated with afirst communication apparatus, and those blocks are also transferredbetween the first communication apparatus and a second communicationapparatus over a further communication link. The communication trafficblocks and further communication traffic blocks that are communicatedover a second wireless communication link associated with the secondcommunication apparatus together comprise communication traffic of thesame communication traffic stream. Communicating the communicationtraffic blocks may involve transmitting from one apparatus communicationtraffic blocks that are transferred from another apparatus, as shown at172, 174, or transferring to another apparatus communication trafficblocks that are received at one apparatus over a local wirelesscommunication link, as shown at 176, 178. The communicating andtransferring operations may thus be performed in different orders,depending on whether communication traffic blocks are received or are tobe transmitted over the wireless communication link.

The methods 140, 160, 170 are illustrative of embodiments of theinvention. Other embodiments may involve further, fewer, and/ordifferent operations, performed in a similar or different order.Additional variations may also be or become apparent to those skilled inthe art. For example, various options for performing some of theoperations shown in FIGS. 6 to 8, as well as other variations, will beapparent from the foregoing description of FIGS. 3 to 5.

In accordance with embodiments of the invention, multiple wireless linksmay be interconnected to deliver higher aggregate bandwidth whilefeeding traffic to and from a network through a primary link. Theinterconnection may be through Ethernet or other media. Communicationtraffic is processed, and radio blocks can be generated independent ofthe source or destination and other information that appears in theheader or overhead of such traffic in order to allow separate trafficstreams to be differentiated. The generated radio blocks, which areuniquely identified in some embodiments, are distributed between primaryand secondary apparatus for transmission over primary and secondarywireless communication links. The interconnection between the primaryand secondary apparatus is used by the primary apparatus to send radioblocks to and receive radio blocks from the secondary apparatus. Thesame interconnection can also be used to carry management traffic, inwhich case a mechanism such as packet tagging could be used to keep themanagement traffic separated from the radio blocks.

Radio blocks that are received by the primary and secondary apparatusover the aggregated wireless communication links are combined torecreate communication traffic streams in the primary apparatus forfurther transmission in a core or access network. As noted above, radioblocks are generated sequentially in some embodiments, and sequencenumbers can be used in the radio blocks to maintain their correctsequential order when rebuilding originally received communicationtraffic from which those radio blocks were generated.

The bandwidth aggregation techniques proposed herein enable higherbandwidth to be provided by using multiple lower bandwidth wirelesscommunication links. All communication traffic is processed in theprimary apparatus, which in some embodiments ensures that the traffic isload balanced over the multiple wireless communication links independentof the actual content of the traffic.

A primary apparatus could also track secondary apparatus functionalinformation to determine the availability of the secondary linkbandwidth and radio operational characteristics. The primary andsecondary apparatus may operate at the same or different wirelessthroughputs. A measure of redundancy protection may also be provided. Awireless error monitoring capability for the secondary link(s), forexample, may enable faster rerouting of radio blocks when a secondarywireless communication link fails. Such monitoring might actually beimplemented in a secondary apparatus or in a primary apparatus. Theprimary apparatus could potentially run multiple monitors, including onefor its own wireless communication link and another for each secondarywireless communication link. The block combiner 78 (FIG. 3), forinstance, could declare an error or failure of a secondary wirelesscommunication link when it does not receive any radio blocks from thatlink within a certain period of time.

What has been described is merely illustrative of the application ofprinciples of embodiments of the invention. Other arrangements andmethods can be implemented by those skilled in the art without departingfrom the scope of the present invention.

For example, the division of functions shown in FIG. 3 is illustrativeof an embodiment of the invention. Further, fewer, or different elementsmay be used to implement the techniques disclosed herein. Blockgeneration, distribution, and combining could potentially be performedby a single physical component, for instance.

In addition, although described primarily in the context of methods andsystems, other implementations of the invention are also contemplated,as instructions stored on a computer-readable medium, for example.

1. An apparatus comprising: a block generator that receivescommunication traffic and generates fixed-length communication trafficblocks for transmission over a wireless communication link, thefixed-length communication traffic blocks comprising the receivedcommunication traffic; a distributor, operatively coupled to the blockgenerator, that distributes the fixed-length communication trafficblocks between a first wireless communication link associated with theapparatus and a second wireless communication link associated with afurther apparatus for transmission of the fixed-length communicationtraffic blocks over the first and second wireless communication links;and a block transfer module, operatively coupled to the distributor,that enables the fixed-length communication traffic blocks that aredistributed to the second wireless communication link to be transferredto the further apparatus over a further communication link, at least oneof the block generator, the distributor, and the block transfer modulebeing implemented using hardware.
 2. The apparatus of claim 1, whereinthe block generator receives the communication traffic over acommunication link, and wherein the communication link is a same type ofcommunication link as the further communication link.
 3. The apparatusof claim 2, wherein the communication link and the further communicationlink comprise respective Ethernet links.
 4. The apparatus of claim 3,wherein the block transfer module encapsulates the fixed-lengthcommunication traffic blocks that are distributed to the second wirelesscommunication link into Ethernet frames.
 5. The apparatus of claim 1,wherein the received communication traffic comprises header information,wherein the distributor distributes the fixed-length communicationtraffic blocks between the first wireless communication link and thesecond wireless communication link independently of the headerinformation.
 6. The apparatus of claim 1, wherein the block generatorincludes a unique identifier in each of the fixed-length communicationtraffic blocks.
 7. The apparatus of claim 1, wherein the distributordistributes the fixed-length communication traffic blocks to loadbalance transmission of the received communication traffic over thefirst and second wireless communication links.
 8. The apparatus of claim1, further comprising: an interface, operatively coupled to the blocktransfer module, that enables communications with the further apparatusover the further communication link, wherein the interface furtherenables exchange of management traffic between the apparatus and thefurther apparatus.
 9. The apparatus of claim 1, wherein the distributorfurther determines an operational status of each of the first and secondwireless communication links, and determines a distribution of thefixed-length communication traffic blocks based on the determinedoperational status.
 10. The apparatus of claim 1, wherein the firstwireless communication link comprises a primary wireless communicationlink, wherein the second wireless communication link comprises one of aplurality of secondary wireless communication links, each secondarywireless communication link being associated with a respective furtherapparatus, wherein the distributor distributes the fixed-lengthcommunication traffic blocks between the first wireless communicationlink and the plurality of secondary wireless communication links,wherein the block transfer module enables the fixed-length communicationtraffic blocks that are distributed to the secondary wirelesscommunication links to be transferred to the further apparatus withwhich each of the secondary wireless communication links is associated.11. The apparatus of claim 1, wherein the block generator receives thecommunication traffic over a communication link, wherein the blocktransfer module further enables fixed-length communication trafficblocks received by the further apparatus over the second wirelesscommunication link to be transferred to the apparatus from the furtherapparatus, the apparatus further comprising: a block combiner,operatively coupled to the block transfer module, that combinesfixed-length communication traffic blocks that are received from thefurther apparatus with fixed-length communication traffic blocks thatare received over the first wireless communication link into a combinedcommunication traffic stream for transmission over the communicationlink.
 12. The apparatus of claim 1, further comprising: a block flowcontroller, operatively coupled to the distributor, that controlsdistribution of the fixed-length communication traffic blocks to thesecond wireless communication link by the distributor.
 13. The apparatusof claim 1, implemented in a first node of a communication system, incombination with a second node of the communication system thatcomprises the further apparatus, wherein the first node furthercomprises a wireless interface that enables the fixed-lengthcommunication traffic blocks that are distributed to the first wirelesscommunication link to be transmitted over the first wirelesscommunication link, wherein the further apparatus comprises a blocktransfer module that enables the fixed-length communication trafficblocks that are distributed to the second wireless communication link tobe received by the further apparatus over the further communicationlink, wherein the second node further comprises a wireless interface,operatively coupled to the block transfer module of the furtherapparatus, that enables the fixed-length communication traffic blocksthat are distributed to the second wireless communication link to betransmitted over the second wireless communication link.
 14. Thecombination of claim 13, wherein the further apparatus comprises: ablock generator that receives communication traffic and generatesfixed-length communication traffic blocks comprising the receivedcommunication traffic; and a distributor, operatively coupled to theblock generator and to the block transfer module, that distributes thefixed-length communication traffic blocks between the first and secondwireless communication links for transmission, wherein the blocktransfer module of the further apparatus enables the fixed-lengthcommunication traffic blocks that are distributed to the first wirelesscommunication link by the distributor of the further apparatus to betransferred to the apparatus over the further communication link,wherein the block transfer module of the apparatus enables thefixed-length communication traffic blocks that are distributed to thefirst wireless communication link by the distributor of the furtherapparatus to be received by the apparatus over the further communicationlink, wherein the apparatus of the first node is configurable foroperation in either of a primary operating mode in which the blockgenerator of the apparatus receives communication traffic and generatesfixed-length communication traffic blocks comprising the receivedcommunication traffic and the distributor of the apparatus distributesthe fixed-length communication traffic blocks between the first andsecond wireless communication links, and a secondary operating mode inwhich the block transfer module of the apparatus receives fixed-lengthcommunication traffic blocks that are distributed to the first wirelesscommunication link by the distributor of the further apparatus, whereinthe further apparatus of the second node is configurable for operationin either of the primary operating mode in which the block generator ofthe further apparatus receives communication traffic and generatesfixed-length communication traffic blocks comprising the receivedcommunication traffic and the distributor of the further apparatusdistributes the fixed-length communication traffic blocks between thesecond and first wireless communication links, and the secondaryoperating mode in which the block transfer module of the furtherapparatus receives fixed-length communication traffic blocks that aredistributed to the second wireless communication link by the distributorof the apparatus, wherein one of the apparatus and the further apparatusis configured for operation in the primary operating mode and the otherof the apparatus and the further apparatus is configured for operationin the secondary operating mode, wherein the configurations of theapparatus and the further apparatus are adjustable, to configure the oneof the apparatus and the further apparatus for operation in thesecondary operating mode and to configure the other of the apparatus andthe further apparatus in the primary operating mode.
 15. A methodcomprising: receiving communication traffic at a first communicationapparatus; generating fixed-length communication traffic blocks fortransmission over a wireless communication link, the fixed-lengthcommunication traffic blocks comprising the received communicationtraffic; distributing the fixed-length communication traffic blocksbetween a first wireless communication link associated with the firstcommunication apparatus and a second wireless communication linkassociated with a second communication apparatus for transmission of thefixed-length communication traffic blocks over the first and secondwireless communication links; transmitting the fixed-lengthcommunication traffic blocks that are distributed to the first wirelesscommunication link over the first wireless communication link; andtransferring the fixed-length communication traffic blocks that aredistributed to the second wireless communication link to the secondcommunication apparatus over a further communication link.
 16. Themethod of claim 15, wherein the further communication link comprise anEthernet link, and wherein transferring comprises encapsulating thefixed-length communication traffic blocks that are distributed to thesecond wireless communication link into Ethernet frames.
 17. The methodof claim 15, wherein the received communication traffic comprises headerinformation, wherein distributing comprises distributing thefixed-length communication traffic blocks between the first wirelesscommunication link and the second wireless communication linkindependently of the header information.
 18. The method of claim 16,wherein generating comprises including a unique identifier in each ofthe fixed-length communication traffic blocks.
 19. The method of claim15, wherein distributing comprises distributing the fixed-lengthcommunication traffic blocks to load balance transmission of thereceived communication traffic over the first and second wirelesscommunication links.
 20. The method of claim 15, wherein the firstwireless communication link comprises a primary wireless communicationlink, wherein the second wireless communication link comprises one of aplurality of secondary wireless communication links, each secondarywireless communication link being associated with a respective furtherapparatus, wherein distributing comprises distributing the fixed-lengthcommunication traffic blocks between the first wireless communicationlink and the plurality of secondary wireless communication links,wherein transferring comprises transferring the fixed-lengthcommunication traffic blocks that are distributed to the secondarywireless communication links to the further apparatus with which each ofthe secondary wireless communication links is associated.
 21. The methodof claim 15, wherein receiving comprises receiving the communicationtraffic over a communication link, the method further comprising:receiving at the first communication apparatus, from the secondcommunication apparatus over the further communication link,fixed-length communication traffic blocks received by the secondcommunication apparatus over the second wireless communication link;receiving fixed-length communication traffic blocks over the firstwireless communication link at the first communication apparatus; andcombining fixed-length communication traffic blocks that are receivedfrom the second communication apparatus with fixed-length communicationtraffic blocks that are received over the first wireless communicationlink into a combined communication traffic stream for transmission overthe communication link.
 22. The method of claim 15, wherein the firstcommunication apparatus, when configured in a primary operating mode,performs the receiving, generating, distributing, transmitting, andtransferring, wherein the second communication apparatus, whenconfigured in the primary operating mode, receives communicationtraffic, generates fixed-length communication traffic blocks comprisingthe received communication traffic, distributes the fixed-lengthcommunication traffic blocks between the second and first wirelesscommunication links for transmission, transmits the fixed-lengthcommunication traffic blocks that are distributed to the second wirelesscommunication link over the second wireless communication link, andtransfers the fixed-length communication traffic blocks that aredistributed to the first wireless communication link to the firstcommunication apparatus over the further communication link, wherein thefirst communication apparatus, when configured in a secondary operatingmode, receives fixed-length communication traffic blocks that aredistributed to the first wireless communication link by the secondcommunication apparatus over the further communication link andtransmits the received fixed-length communication traffic blocks overthe first wireless communication link, wherein the second communicationapparatus, when configured in the secondary operating mode, receivesfixed-length communication traffic blocks that are distributed to thesecond wireless communication link by the first communication apparatusover the further communication link and transmits the receivedfixed-length communication traffic blocks over the second wirelesscommunication link, the method further comprising: configuring one ofthe first and second communication apparatus for operation in theprimary operating mode; configuring the other of the first and secondcommunication apparatus for operation in the secondary operating mode;and adjusting the configurations of the first and second communicationapparatus, to configure the one of the first and second communicationapparatus for operation in the secondary operating mode and to configurethe other of the first and second communication apparatus in the primaryoperating mode.
 23. An apparatus comprising: an interface that enablesreception of fixed-length communication traffic blocks over a firstwireless communication link associated with the apparatus; a blocktransfer module that enables fixed-length communication traffic blocks,that are received by a further apparatus over a second wirelesscommunication link associated with the further apparatus, to betransferred to the apparatus from the further apparatus over a furthercommunication link; and a block combiner, operatively coupled to theinterface and to the block transfer module, that combines fixed-lengthcommunication traffic blocks that are received from the furtherapparatus with fixed-length communication traffic blocks that arereceived over the first wireless communication link into a combinedcommunication traffic stream.
 24. A method comprising: receivingfixed-length communication traffic blocks over a first wirelesscommunication link associated with a first communication apparatus;receiving at the first communication apparatus, from a secondcommunication apparatus over a further communication link, fixed-lengthcommunication traffic blocks that are received by the secondcommunication apparatus over a second wireless communication linkassociated with the second communication apparatus; and combiningfixed-length communication traffic blocks that are received from thesecond communication apparatus and that comprise communication trafficof a communication traffic stream with fixed-length communicationtraffic blocks that are received over the first wireless communicationlink and that comprise communication traffic of the communicationtraffic stream into a combined communication traffic stream.